Simulating Artworks through Filter Blending

This paper looks at a method of blending different artistic filters together to create a range of artistic effects. Instead of using a single painterly rendering technique, the methods used in three different filters can be blended together in a user defined way. The filters are arranged in a triangular structure where the user defines their chosen painting style by choosing a point in the triangle. This allows users to effectively create their own painting style and experiment with a range of different artistic effects. The field of painterly rendering looks at methods of creating a simulated artwork from a source photograph. Artistic filters are inspired by methods used by real artists. Most of these filters look at a particular aspect of a real painting process and design an algorithm to simulate this process. Our contribution takes inspiration, not from one aspect of a painting process, but from the variety that is found in painting methods. We look at ways to blend aspects of different filters together to create a unique artistic effect that is chosen by the user. We use three different artistic filters in this paper. The first is Aaron Hertzmann’s painting algorithm described in [2]; it uses layers of curved brush strokes. Next is a pointillistic filter roughly based on [5] that attempts to simulate the works of Georges Seurat. The last is Papari and Petkov’s method for creating impressionist paintings using Glass patterns as described in [3]. Sections 1, 2 and 3 look at the methods used in each of the different artistic filters. Section 4 described our methods for blending these filters together. Section 5 shows the results of this blending technique and Section 6 discusses our conclusions and ideas for future work. 1 Curved Brush Strokes This section describes an algorithm presented by Aaron Hertzmann that simulates a layered painting style with brush strokes of varying sizes. Hertzmann’s algorithm uses a layered approach; it starts with a rough approximation of the image and builds up more detail at each layer with steadily smaller brush strokes. These are curved strokes that follow object contours, as favored by many artists. The results of this process can be seen in Fig. 1. Layering of Strokes. We start with a blank canvas and a reference image. The algorithm takes an array of difference brush sizes as a parameter. Each brush size defines a layer in the painting; starting with the largest brush and working down to smallest, defined by a minimum brush size bmin and maximum size bmax and equidistant values in-between. 2 Crystal Valente and Reinhard Klette Fig. 1. Left: Scene from Fiji. Original photograph by Marian Arnold. Right: The results of the curved strokes algorithm. For each brush size we divide the image into a grid where the size of each cell is proportional to the current brush size. At each grid point we determine the total error of each pixel contained in this grid area by comparing the color of the canvas at this point to the color of the reference image. If the total error is above a threshold T , we add a new stroke to the canvas at the point in the neighborhood with the maximum error. Threshold T determines how closely the finished painting approximates the reference image and therefore how loose our painting style will be. We do not want the grid structure to make our strokes look too uniform, so strokes need to be painted in a random order. This layering process can be applied to strokes of any shape and orientation. Curved Brush Strokes. We describe the creation of the curved brush strokes. Hertzmann’s algorithm for placing strokes is as follows. The process takes as input a brush radius R, a starting point (x0, y0), and a maximum length L. We start at the point (x0, y0) and find the color value C at this point in the reference image. C gives us the color of the stroke that remains constant. We paint a circle of radius R and color C at point (x0, y0) to the image canvas. The next point in the stroke is computed by finding the normal to gradient at this point. We find the direction of the gradient θ by determining the convolution of the Sobel operator with the luminance of the reference image in the x and y directions. The next point in our spline (x1, y1) is placed distance R from point (x0, y0) in direction θ + π2 . This point is also a circle with radius R and color C. This process is repeated until all the control points in the stroke have been painted to the canvas. The process terminates when (a) the user defined maximum stroke length L is reached, or (b) the color of the stroke differs from the color of the reference image at the last control point more than it differs from the image canvas at that point.